Partial least squares-based human upper body orientation estimation with combined detection and tracking

a r t i c l e i n f o Keywords: Human upper body orientation Partial least squares Multi-level HOG-LBP Random Forest classifier UKF tracker This paper deals with the problem of estimating the human upper body orientation. We propose a framework which integrates estimation of the human upper body orientation and the human movements. Our human orientation estimator utilizes a novel approach which hierarchically employs partial least squares-based models of the gradient and texture features, coupled with the random forest classifier. The movement predictions are done by projecting detected persons into 3D coordinates and running an Unscented Kalman Filter-based tracker. The body orientation results are then fused with the movement predictions to build a more robust estimation of the human upper body orientation. We carry out comprehensive experiments and provide comparison results to show the advantages of our system over the other existing methods. Human body orientation estimation is one of challenging yet useful tasks for the mobile robot and surveillance applications. The human body orientation can tell us how people interact with each other in the surveillance scenes. For example, we may predict that a group of persons facing each other for a long time are having conversation, or other social semantic predictions such as waiting for the bus together. From the mobile robots point of view, the human body orientation can assist the robot to get a better prediction for avoiding a person when doing navigation tasks. It also helps the robot to make a social interaction with the human in outdoor navigation, such as approaching a person and asking the way. Here the robot certainly needs the estimation of human orientation for facing the person. Several works try to accomplish the human body orientation problem. Andriluka et al. [3] use banks of viewpoint specific part based detectors and linear Support Vector Machine (SVM) for estimating the whole body orientation. Another recent work is done by Weinrich et al. [4] which performs the human upper body orientation estimation using Histogram of Oriented Gradient (HOG) features and SVM Tree. A work by Baltieri et al. [21] employ a Mixture of Approximated Wrapped Gaussian (MoAWG) weighted by the detector outputs for increasing the correct estimation rates. The above researchers do not consider how the person movements will affect the overall estimation results. One notable work by Chen et al. [5] uses multi-level HOG features and sparse representation for …